The asynchronous transfer of messages between application programs running different data processing systems within a network is well known in the art, and is implemented by a number of commercially available messaging systems. These systems include IBM Corporation's WebSphere MQ family of messaging products, which use asynchronous messaging via queues. A sender application program issues a PutMessage command to send (put) a message to a target queue, and a WebSphere MQ queue manager program handles the complexities of transferring the message from the sender to the target queue, which may be remotely located across a heterogeneous computer network. The target queue is a local input queue for another application program, which retrieves (gets) the message from this input queue by issuing a GetMessage command asynchronously from the send operation. The receiver application program then performs its processing on the message, and may generate further messages. (IBM and WebSphere are trademarks of International Business Machines Corporation).
Messaging can be persistent or non-persistent. In non-persistent messaging there is no assurance that the message will be delivered. In persistent messaging there is an assurance of delivery and in one technique, this is achieved by storing the message (often referred to as “hardening”) to non-volatile memory (e.g. disk storage device) as part of the messaging process. This means that if the messaging system fails during the message transfer then the message can be recovered. Although persistent messaging prevents loss of messages, there is a significant overhead involved in hardening to disk. Where applications are not concerned about recovery of lost messages then non-persistent messaging is employed where the message data is not hardened to disk and the overhead involved in hardening is avoided.
Messaging can be transactional or non-transactional. Transaction processing is a well known technique and in general terms involves the execution of discrete units of work that access and update shared data. Logical points of consistency at which resource changes are synchronised within transaction execution are called commit points or syncpoints. An application ends a unit of work by declaring a syncpoint, or by the application terminating. The characteristic of a transaction being accomplished as a whole or not at all is known as ‘atomicity’. Atomicity of a transaction is known to be achieved by resource updates made within the transaction being held uncommitted until a syncpoint is declared at completion of the transaction. That is, the resource updates are only made permanent and visible to applications other than the one which performed the updates on successful completion. If the transaction fails to complete successfully, then all changes that have been made to resources during the partial execution are removed—the transaction is said to rollback (or synonymously to backout), the resources being restored to the consistent state which existed before the transaction began. Any party (e.g. an application or resource manager) with an interest in the unit of work can cause a rollback when a syncpoint is declared by indicating unreadiness to commit.
In one example of transactional processing as applied to persistent messaging, a message is PUT ‘inside’ the syncpoint and when COMMITTED it is hardened to disk. A number of messages may advantageously be batched inside a single syncpoint and hardened to disk as a batch thus improving the efficiency of the operation. In one example of non-transactional persistent messaging, a message is PUT ‘outside’ syncpoint. When there is a corresponding GET waiting, the message will be transferred to the recipient thus resulting in an efficient message transfer.
Examples of transactional and non-transactional persistent messaging will now be described in more detail with reference to FIGS. 1 and 2. FIG. 1 represents an example of persistent messaging between Application A 10 and Application B 20 where a message is outside syncpoint. First, Application A PUTS 50 a message onto the queue controlled by queue manager 30. Because there is no corresponding GET from Application B already on the queue and the message is defined to be persistent and outside syncpoint, the PUT is immediately hardened onto non-volatile storage, represented in FIG. 1 by disk 100. The queue manager indicates 51 to Application A that the message has been PUT on the queue. At some later time, Application B sends 53 a GET to the queue for the message previously PUT there by Application A. As the GET is outside syncpoint, the Application A message is retrieved from the queue, hardened to disk and transmitted to Application B. Thus because there is no outstanding GET at the time of the PUT, this non-transactional message transfer involves two disk hardening operations for a single message transfer thus reducing the efficiency of the system.
If, as previously indicated, at the time the PUT from Application A was received by the queue manager, a corresponding GET from Application B had already been on the queue, the message could have been immediately transferred to Application B, thus avoiding the two hardening operations.
FIG. 2 represents an example of persistent messaging where the PUT is inside a syncpoint. First, Application B sends 55 a GET WAIT NO_SYNC to the queue manager. This GET waits on the queue for a corresponding PUT. In step 56, Application A PUTS the corresponding message on the queue. Because this message is inside a syncpoint, it is not visible to the GET until it has been committed and thus the message cannot be transferred to Application B. In the meantime, Application A issues a number of PUTs 56a,b,c inside syncpoint and finally COMMITS at step 57. At this point the four messages are hardened to disk which causes message 56 to become available to the corresponding GET 55. The removal of the message 55 is hardened to disk and the message 55 is transferred to Application B. Thus, whilst the disk hardening overhead has been decreased by PUTTING multiple messages inside syncpoint, the transfer of the initial PUT 55 to Application B has taken longer than if PUT 56 had been placed outside syncpoint.
As the sending application will invariably be unaware as to whether there is a GET waiting on the queue for a PUT it is about to send, it can inadvertently choose the less efficient syncpoint option for a particular message. In particular, when there is no outstanding GET it would be preferable to place the message inside syncpoint as placing it outside syncpoint would result in disk hardening operations for each PUT. On the other hand, when there is an outstanding GET, it would be preferable to place the message outside syncpoint as placing it inside syncpoint would result in unnecessary delay in message transfer. It would be desirable to ameliorate one or more of these disadvantages.